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(54) System and method for embedding a watermark in digital image sequences 



(57) A method for embedding a watemnark signal 
that contains message data in a digital Image sequence 
having two or more frames, includes the steps of: pro- 
ducing a set of two or more different cannier signals from 



a single secure key; assigning a carrier signal from the 
set of carrier signals to each frame In the sequence; and 
embedding a watenrnark signal in each frame using the 
corresponding assigned carrier signal. 
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Description 

[0001] The invention relates generally to the field of digital image processing, and in particular to a method for em- 
bedding watemriarks In digital image sequences. 

5 [0002] Digital watermarking refers to the embedding of a hidden message in an image or image sequence for such 
purposes as establishing ownership, tracking the origin of the data, preventing unauthorized copying, or conveying 
additional Infomiation (meta-data) about the content. Watemnarking has potential uses in a wide range of products, 
including digital still and video cameras, printers and other hardcopy output devices, and content delivery services (e. 
g., Internet-based photofinlshing). Recently, there has been significant interest in the electronic distribution and display 

10 of theatrical movies, which is termed digital cinema. Studios and distributors have a strong need to protect the movie 
content from unauthorized use, and watermarking can assist by establishing ownership and tracing the source of stolen 
content (through the use of hidden dateAime/tocation stamps inserted at the time of the movie distribution and/or 
presentation). The present invention relates specifically to the watermarking of image sequences, and thus It has 
usefulness in an application such as digital cinema. 

15 [0003] Numerous watermarking methods have been described In the prior art, including both patents and the tech- 
nical literature. Many of these methods are described in review papers such as: Hartung and Kutter, Multimedia Wa- 
temnarking Techniques, Proc. IEEE, 87(7), pp. 1079-1107 (1999), and Wolfgang et aL, Perceptual Watermarks for 
Digital Images and Video, Proc. IEEE. 87(7), pp. 1108-1126 (1999). 

[0004] A basic distinction between various methods is whether the watermark is applied in the spatial domain or the 

20 frequency domain. In either approach, many techniques make use of a pseudo-random (PN) sequence in the watermark 
generation and extraction processes. The PN sequence serves as a carrier signal, which is modulated by the original 
message data, resulting In dispersed message data (i.e., the watemnark) that is distributed across a number of pixels 
in the image. A secret key (i.e., seed value) is commonly used in generating the PN sequence, and knowledge of the 
key is required to extract the watermark and the associated original message data. 

25 [0005] As noted In the review papers by Hartung et al. and by Wolfgang et al., most research on watermarking 
techniques has focused on single-frame images, and there are signlfbantly fewer methods that are specific to image 
sequences (i.e., video watermarking). Of course, a watermarking method that has been designed for single-frame 
images could be applied to an image sequence by merely repeating the same process for each frame. However, this 
approach has the disadvantage that the fixed watermark pattem may become perceptually objectionable when the 

30 image sequence is displayed in real-time. 

[0006] There are several prior art patents that include video-specific watemnarking methods: U:S. Patent 5,809.139 
issued September 15,1 998 to GIrod et al. entitled Watermarking Method and Apparatus for Compressed Digital Video, 
B. GIrod et al., Sept. 15, 1998; U.S. Patent 5.901 .178 issued May 4, 1999 to Lee et al. entitled Post-Compression 
Hidden Data Transport for Video; U.S. Patent No. 5,991,426 issued November 23, 1999 to Cox et al. entitled Fleld- 

35 Based Watemnark Insertion and Detection; and U.S. Patent No. 6.026,1 93 issued February 15, 2000 to Rhoads entitled 
Video Steganography. 

[0007] In the patents by Girod et al. and Lee et al., the methods are designed for directly embedding a watermark 
in compressed frequency-domain yideo streams (such as MPEG-encoded sequences). The patent by Cox et al. de- 
scribes a method for alternately embedding positive and negative watermarks in consecutive fields of an interiaced 

40 video signal; this method is not suitable for progressively scanned image sequences such as those used in digital 
cinema applications. The patent by Rhoads discloses the basic concept of using multiple watemnarked frames from 
an image sequence to extract the watermark with a higher degree of confidence than would be obtained with only a 
single frame. However, the methods described in all of the aforementioned patents make use of the same watermarking 
pattem in each successive frame of the sequence. As a result, these methods are subject to the same disadvantage 

45 as previously mentioned, namely, the presence of a fixed watermark pattern that can be objectionable. 

[0008] There are obvious modifk:atlons that can eliminate the fixed watemnark pattern, but they also suffer from 
limitations. One modification is to change the PN carrier from frame to frame by using a different key for each frame. 
However, the management of numerous keys becomes problematic. Another modification is to change the message 
while using the same carrier signal, but it may not be desirable to change the message from frame to frame In many 

so applications. Moreover, changing the message from frame to frame does not allow Information from multiple frames 
to be combined when extracting the watermark. This limitation reduces the robustness of the watermark extraction 
process to certain types of removal attacks. 

[0009] There is a need therefore for an Improved watermarking technique that: (1 ) minimizes the visibility of the 
watemnark when the watennarked sequence is displayed In real-time; (2) requires only a single key for the generation 
S5 and extraction of the watemnark data; and (3) allows for Information from multiple frames to be combined when ex- 
tracting the watermark. 

[0010] The need Is met according to the present invention by providing a method for embedding a watermark signal 
that contains message data In a digital image sequence having two or more frames, including the steps of: producing 
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a set of two or more different carrier signals fronn a single secure key; assigning a carrier signal from the set of carrier 
signals to each frame in the sequence; and embedding a watermark signal In each frame using the corresponding 
assigned carrier signal 

[0011] The present invention minimizes the visibility of a watermark in an image sequence while simultaneously 
5 providing the convenience of a single-key system. The invention also allows watenmark infonnation to be combined 
from multiple frames, whbh improves the robustness of the watemnark extraction process. 

Fig. 1 is a prior art method for embedding a watermark in an origirial Image; 

Fig. 2 is a prior art method for extracting a watermark from an image containing an embedded watermark; 
10 Fig. 3 illustrates the generation of multiple carriers from a single key and the application of the multiple carriers to 

a sequence of frames in the present invention; 

Fig. 4 illustrates the detenniriatlon of the carrier for a gh/en frame using a search over the set of possible carriers; 
Fig, 5 illustrates the generation of multiple carriers from a single key by segmenting a larger carrier image into 
regions; 

f 5 Fig. 6 illustrates the segmentation of a larger carrier image into smaller carrier images by f omiing non-overlapping, , 

contiguous regions; 

Fig. 7 illustrates the generation of multiple carriers from a single key by spatially transfomning a carrier image; 
Fig. 8 Illustrates the generation of multiple carriers from a single key by fonning cyclical shifts of the key; and 
Fig. 9 Illustrates the process for combining the extracted messages from multiple frames. 

20 

[0012] The present invention overcomes the limftations of the prior art by using a single key to generate multiple 
carriers for the watermarking of an image sequence. The use of multiple carriers minimizes the visibility of a watermark 
by preventing spatial alignment of the same watermark pattern from frame to frame. However, unlike systems where 
different secret keys are used in generating the carriers, the present invention uses only a single key in combination 

25 with deterministic transformations of eitherthe key orthe associated camerto produce multiple carriers. The robustness 
of the watermark extraction process is unchanged when It is applied to a single frame, b^ecause the same carrier is 
used within a given frame. However, because the same message is embedded in each frame^ it is possible to combine 
infonnation from multiple frames after determination of the carrier signal that was used for each frame. The present 
invention is aimed primarily at watermaric methods that embed in the spatial domain using a PN sequence in producing 

30 a carrier signal. However, it can also be applied to frequency domain methods that use a PN sequence in the water- 
marking process. 

[0013] The present invention is preferably implemented by a programmed digital computer. The computer can be a 
general purpose digital computer or a special purpose computer for digitaj iniage processing, it is within the ordinary 
skill in the programming art to provide a computer program for practicing the present Invention from the followirig 

35 description of the invention. 

[0014] A prefenred data embedding technk^ue for use with the present invention Is disclosed in U.S. Patent No. 
6.044,156 issued March 28, 2000 to Honsinger et al enmed Method for Generating an Improved Canier for Use in an 
image Data Embedding Application, This patent is included in its entirety by reference. Referring to Fig. 1, in this 
technique, an original two-dimensional image 10, /(x,y), is processed to produce a watermarked image 12, t{x,y). A 

40 two-dimensional message 14, M(x,y), represents the data to be embedded in the original image. In its most general 
fomi, the message 14 is an image, and it can represent an Icon 16 (e.g., a trademark), or it can represent the bits 18 
in a binary message. In the latter case, the on and off states of the bits are represented as plus and rhirius ones (rfibre * 
specifically, positive and negative delta functions), which are placed in predefined and unique locations across the 
message image. Examples of iconic message data are trademarks, corporate logos or other arbitrary iniageis. In order ' 

45 to minimize the message energy, an edge map of the icon is often used instead oif the actual icon. EXanhples of binary 
message data are 32-bit representations of URL's, and copyright ID codes, or authentication infonnation. 
[0015] As shown in Fig. 1 , the fundamental steps for embedding message data in an original image with this method 
are: 

^o 1. A fj x n message image 14, /l^x,>0, is generate^d from the message data; 

2. The message image 14 is .circulariy convolved 20 with a n x n carrier Image 22, C(x,y), to produce a n x n 
dispersed message image 24. The carrier image may be produced using a secure key 26 as is known in the prior art; 

3. The dispersed message image 24 is scaled 28 in amplitude using a multiplicative factor a ; and 

4. The scaled dispersed message image 30 is added to the original image 10 as contiguous n x n tiles to form a 
55 watermarked image 12, /"(x,/). 

[0016] The tiling of the dispersed message images forms the watemnark pattem that is combined with the original 
image. The scaling factor a is an arbitrary constant chosen to make the watermark pattem simultaneously invisible • 
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Cj(x,y)®S(x,y) = Cj(x,y)®[C|(x,.y)-hC2(x,y)4-K +Cj(x,y)+K -f CN(x,y)] 
-CjCx,y)®Cj(x,y) 

(7) 

where Cj(x,y) denotes the carrier image used for the frame. Thus, the use of the super carrier during the extraction 
process is equivalent to using only the correct canrier Image for a given frame as described previously in Fig. 2. Of 
course, the can-ler images must be carefully designed to Insure that they are uncorrelated as any con-elation will con- 
tnbute to the noise temi shown in Eq. 4. which can lead to reduced robustness In the extraction process. It Is noted 
that although the present Invention uses a single key to generate multiple carriers, the aforementioned method for 
extraction using the "super carrier" will also work for a system where multiple carriers are generated from multiple keys 
(again assuming that the carriers are uncorrelated). 

[00271 The generation of multiple carrier Images from a single key can be perfomied by a number of different means 
Regardless of the specific mettiod, the canler Images that are produced by a single key are con-elated In a cryptographic 
sense (but not necessarily In a statistical conflation sense), which can lead to concerns about the Inherent security of 
this approach. However, one can easily increase the key length to overcome most of these concems. More Importantly, 
from a perceptual point-of-view, the carrier images are Independent realizations, and the resulting watermark patterns 
will be entirely different to an observer, even though a single key was used in their generation. 

[0028] As shown In Fig. 5, in a prefen-ed embodiment, the single key 26 is used to generate 50 a earner image C(x, 
y) of size m x m. where m is chosen to be larger than the actual earner image 22 that is used in the watermartcing 

process (i.e., the embedded tile size is n x n, where n < m). The different carrier images CXx,y). (/= 1 AJ) can be 

produced from the larger carrier image, C(x.y), by simply segmenting 52 C(x.y) into N different nxn r^ions. The 
segmentation 52 of C(x,y) can be done in a variety of ways, but the most straightfonward Is to choose mto be an integer 
multiple of n and segment Qx.y) into non-overlapping, contiguous regions as shown in Fig. 6. 

[0029] As shown in Fig. 7, in another pretended embodiment, the single key 26 is used to generate 54 a carrier image 
C(x,y) of the same size (n x n) as the actual can-ier used in the watemriarking process. Different carrier images are 
then fomned by spatially transforming 56 this initial earner image C(x.y). The spatial transformations can include, but 
are not limited to: rotations around the carrier image center at 90« increments; rotations around the horizontal . vertical 
or diagonal axes of the canrier image; and reordering the lines and/or columns of the earner image in some pre-deter- 
mined manner. This approach is somewhat more constrained than the previous embodiment In that there may be a 
limited number of spatial rean-angements that are sufficiently different in a perceptual and/or statistical con-elation 
sense 



[0030] As Shown in Fig. 8, in still another prefen^ed embodiment, the single key 26 is cyclically shifted 58 by different 
numbers of bits to produce A/ different keys. Each different key is then used to generate 54 a different nx n earner 
image. A cyclical shift of the initial key by just one bit is suffteient to produce a completely different carrier image. As 
noted previously, the different keys (and resulting carrier Images) produced by this method are cryptographfcally cor- 
related, but the initial key length can be Increased to provide a levet of security that Is comparable to that provided bv 
the use of N Independent keys. 

[0031] Although different carrier images are used for each frame in the sequence, the same message image is used 
for all frames. As a result, the extracted message images from any number of frames can be combined after determining 
the earner images of each frame. This process is Illustrated in Fig, 9, where the correct carrier image for each frame 
IS determined 48, and then the extracted message Images for the frames are combined 64. Moreover, if there are 
frames In the sequence that use the same carrier image (e.g. every AA^ frame might use the same can-ier Image), It is 
also possible to directly combine the tiles from these frames and then extract the message from the average of the 
combined tiles. In either approach, combining the infonnation from multiple frames provides a more robust estimate 
by increasing the signaMo-nolse ratio of the extracted message, which can improve perfonnance under certain types 
of removal attacks and/or allows for the amplitude of the watemnaric to be reduced to a lower level. Reducing the 
amplitude further reduces the visibility of the watermark. 

[0032] In some applications of the present invention, it may be desirable to use the same carrier image for several 
consecutive frames, rather than changing the carrier image with each frame. This may provide additional robustness 
to the watermark extraction process when the image sequence data has been modified during certain types of attacks. 
For example, if a video camcorder is used to capture an Illegal copy of a projected movie in a theater, there is a mismatch 
of the temporal sampling rates of the projected image (24 progressive frames per second) and the video camcorder 
(60 interlaced fields per second). If the watemiark pattern is changed with each frame, there will be occasions when 
the camcorder will integrate different watemnark pattems over two frames. By allowing the same watermaric pattern to 
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persist for two frames, there is an increased probability that the watermark can be extracted from any field or interlaced 
frame of the lilegat video copy. Of course, increasing the display duration of the same watermark pattern beyond two 
frames might further increase the robustness of the extraction process, but the slowty changing watermark pattem will 
also be more easily perceived than one that Is changing every frame or every other frame. 

5 [0033] While the invention has been discussed in terms of the spatial domain watermarking process as described 
by Honsinger et al., It is obvious how the same method can be applied to any spatial domain watermarking process 
that uses a earner image in the watermarking process. It Is Importarit to note that some spatial domain watermarking, 
processes use more than one carrier imajge within a single image, where the resulting watermark pattem is the sum 
of the carrier images after modulation by the message information. An exarhple of such a watermarking process can 

10 be found in U.S. Patent 5,636,292 "Steganography methods employing embedded calibration data" by Rhoads. Al- 
though the method described in the aforementioned patent also uses multiple carrier images, this method is funda- 
mentally different from the present invention in two ways. First, the multiple carrier Images that are used in the method 
by Rhoads are employed within a single image, not across multiple frames in a sequence. Second, the set of carrier 
images is fixed, thus producing a given watermark pattem that is constant from frame to frame. It |s entirely possible 

IS to use the present invention in conjunction with the method of Rhoads when watennarking an image sequence by 
changing one or more of the carrier Images from frame to frame, Again, a single key can be used in generating the 
different carriers, using one or more of the methods described previously. 

[0034] Although the present Invention has been described using the preferred data embedding and extraction meth- 
ods of Honsinger et al. that use two-dimensional carrier images, it is noted that the same concepts can be applied to 
20 other watemnarking methods that use one-dimensional carrier signals. For example, some types of frequency domain 
watermarking methods for images use a PN sequence In the watermarking process. A different PN sequence can be 
generated for each frame using one or more of the previously described methods. 

2s Claims 

1 . A method for embedding a watermark signal that contains message data in a digital image sequence having two 
or more frames, comprising the steps of: 

^0 a) producing a set of two or more different carrier signals from a single secure key; 

b) assigning a carrier signal from the set of carrier signals to each frame in the sequence; and 

c) embedding a watermark signal in each frame using the corresponding carrier signal assigned in step (b). 

2. The method claimed in daim 1, wherein the carrier signals are random phase carrier images, and wherein the 
35 embedding step includes the steps of: 

c1) creating a message image representing the message data; 

c2) convolving the message image with each carrier image to produce a sequence of dispersed message 
images; and 

^o, c3) combining the dispersed message images with the respective frames of the digital image sequence. 

3. The method claimed in claim 1 , wherein the set of carrier signals Is produced by cyclically shifting the secure key 
for each carrier signal; 

45 4. The method claimed In claim 2, wherein the.random phase carrier Images are produced by cyclically shifting the 
secure key for each carrier Image. 

5, The method claimed in claim 2, wherein the random phase carrier images are produced by the steps of: 

so a1) generating a large random phase carrier image; and 

a2) segmenting the large random phase carrier image into smaller random phase carrier images to produce 
the set of random phase carrier images. 



55 



6. The method claimed In claim 2, wherein the random phase carrier images are produced by the steps of: 
a1) generating an initial random phase carrier image; and 

a2) spatially transforming the initial random phase carrier image to produce the set of random phase carrier 
images. 
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7. The method claimed in claim 1 , further comprising the step of extracting rriessage data from at least one frame of 
the image sequence. 

8. The method claimed in claim 7» wherein the message data is extracted by producing a super earner signal by 
5 summing the set of carrier signals and using the super carrier signal to recover the message data. 

9. The method claimed in claim 2, further comprising the step of extracting the message image from at least one 
frame of the image sequence. 

10 10. The method claimed in claim 9, wherein the message image is extracted by producing a super carrier image by 
summing the set of carrier images and correlating the super carrier image with the image sequence frame. 
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